Joint Firearm Training Systems and Methods

ABSTRACT

At least one shooter-side image sensor captures images of a plurality of shooters and a plurality of respective firearms periodically fired by the shooters. At least one target-side sensor collects data indicative of projectile strikes on a target area associated with at least one target. A processing unit analyzes images captured by the shooter-side image sensor and detects projectile discharges in response to firing of the firearms, and uniquely identifies each of the shooters associated with the detected projectile discharges. The processing unit detections of projectile strikes, based on the data collected by the target-side sensor, and the detected projectile discharges and identifies, for each detected projectile strike on the target area, the correspondingly fired firearm associated with the uniquely identified shooter.

TECHNICAL FIELD

The present invention relates to firearm training.

BACKGROUND OF THE INVENTION

In firearm training facilities, such as shooting ranges, in whichmultiple shooters fire simultaneously on respective dedicated targets ora single shared target, tracking of individual shooter performance isparamount. Shooting ranges which utilize non-live fire ammunition, forexample, laser-based firearm training, may provide comprehensiveindividualized shooter performance data, as the firearm munitiondischarges may be laser pulses which can be easily correlated by acomputer system with detection of corresponding laser pulse strikes onthe target. However, conventional shooting ranges that utilize live fireammunition cannot provide simple means for linking individual shooterswith their individual performance. More complicated means exist byproviding the shooters with body mounted detection mechanisms, however,such means can be cumbersome for the shooter.

Combat training simulations, such as those employed by, for example,military, law enforcement and security organizations, may provide meansfor comprehensive performance data. Combat training simulations aretypically divided into the main categories of live simulations,constructive simulation, and virtual simulations. In live simulations,real people operate real systems in non-operational modes, for example,the operation of real firearms using laser pulses instead of live fireammunition. In constructive simulations, simulated people operatesimulated systems, for example military style war games, in which realpeople may make inputs to the simulation, in other words—command,control and decision-making processes, but cannot directly take actionsto effect immediate outcomes. In virtual simulations, real peopleoperate computer-based simulated systems, for example, virtual shootingsimulators. Live simulations and virtual simulations can provide meansfor comprehensive individualized shooter performance data, as livesimulations utilize laser-based ammunition, as discussed above, andvirtual simulations are strictly computer-based which allow computersystems to track individual performance.

However, live fire combat training, which utilizes live fire ammunition,which is typically regarded as the most effective and realistic form ofcombat training, does not provide the basic features of virtual and livesimulations, such as individual shooter performance tracking, andtherefore training involving live fire is usually an undocumentedexperience that does not support methodological learning and efficiencyprocesses over time. Furthermore, live fire combat training does notprovide any means for collaborative training between multiple shootersdeployed in different geographic locations.

SUMMARY OF THE INVENTION

The present invention is directed to systems and methods for jointfirearm training of shooters.

According to the teachings of an embodiment of the present invention,there is provided a system for jointly training a plurality of shooters.The system comprises: a shooter-side sensor arrangement including atleast one shooter-side image sensor deployed to capture images of theshooters and a plurality of respective firearms periodically fired bythe shooters; a target-side sensor arrangement including at least onetarget-side sensor deployed to collect data indicative of projectilestrikes on a target area associated with at least one target; and aprocessing unit including at least one processor operatively coupled tothe sensor arrangements. The processing unit is configured for:analyzing one or more of the images captured by the at least oneshooter-side image sensor to detect projectile discharges in response tofiring of the firearms, and to uniquely identify each of the shootersassociated with the detected projectile discharges, and correlatingdetections of projectile strikes, based on the data collected by the atleast one target-side sensor, and the detected projectile discharges toidentify, for each detected projectile strike on the target area, thecorrespondingly fired firearm associated with the uniquely identifiedshooter.

Optionally, the at least one target-side sensor is an image sensor, andthe processing unit is further configured for analyzing image datacaptured by the target-side image sensor to detect projectile strikes onthe target area corresponding to firing of the firearms.

Optionally, the analyzing one or more of the images captured by the atleast one shooter-side image sensor to uniquely identify each of theshooters associated with the detected projectile discharges is performedby evaluating one or more visual parameters associated with each of theshooters.

Optionally, the at least one shooter-side image sensor has an associatedfield of view, and the field of view is divided into sub-regions, andeach shooter is positioned in a different sub-region.

Optionally, the detected projectile discharges and the detectedprojectile strikes include temporal information.

Optionally, the correlating includes analyzing the temporal informationof the detected projectile discharges and the detected projectilestrikes to form estimated of times of flight for pairs of detectedprojectile discharges and detected projectile strikes.

Optionally, the system further comprises: an image projecting unitoperative to project a virtual scenario onto a background, and thetarget area is included in the projected virtual scenario.

Optionally, each of the shooters has a corresponding virtual entity,projected onto the background by the image projecting unit.

Optionally, the processing unit is linked to a server via a network.

Optionally, the at least one target includes a plurality of targets, andeach respective firearm is fired at a respective one of the plurality oftargets.

Optionally, the plurality of shooters includes more than two shooters, afirst subset of shooters periodically fires a first subset of therespective firearms to strike a first target, and a second subset ofshooters periodically fires a second subset of the respective firearmsto strike a second target, and the first and second targets are deployedin different geographic locations.

Optionally, the at least one shooter-side sensor arrangement includes aplurality of shooter-side sensor arrangements, and the at least onetarget-side sensor arrangement includes a plurality of target-sidesensor arrangement, and the at least one processing unit includes aplurality of processing units, and each of the processing units islinked to a server via a wired or wireless network.

Optionally, a first one of the shooter-side sensor arrangements, a firstone of the target-side sensor arrangements, and a first one of theprocessing units are deployed together in a first geographic location,and a second one of the shooter-side sensor arrangements, a second oneof the target-side sensor arrangements, and a second one of theprocessing units are deployed together in a second geographic locationdifferent from the first geographic location.

There is also provided according to an embodiment of the teachings ofthe present invention a method for jointly training a plurality ofshooters. The method comprises: detecting, based on one or more capturedimages of the shooters and the respectively fired firearms, projectiledischarges in response to firing of a plurality of respective firearmsperiodically fired by the shooters; uniquely identifying, based on theone or more captured images of the shooters and the respectively firedfirearms, each of the shooters associated with the detected projectiledischarges; detecting projectile strikes on a target area associatedwith at least one target corresponding to firing of the firearms; andcorrelating the detected projectile discharges and the detectedprojectile strikes to identify, for each detected projectile strike onthe target area, the correspondingly fired firearm associated with theuniquely identified shooter.

Optionally, the detected projectile discharges and the detectedprojectile strikes include temporal information, and the correlatingincludes analyzing the temporal information of the detected projectiledischarges and the detected projectile strikes to form estimated oftimes of flight for pairs of detected projectile discharges and detectedprojectile strikes.

There is also provided according to an embodiment of the teachings ofthe present invention a system for jointly training a plurality ofshooters periodically firing a plurality of respective firearms. Thesystem comprises: at least one shooter-side sensor arrangement includingat least one shooter-side image sensor deployed to capture images of theshooters; at least one target-side sensor arrangement including at leastone target-side sensor deployed to collect data indicative of projectilestrikes on a virtual target; at least one image projecting unit fordisplaying a virtual environment on a background, the virtualenvironment including the virtual target; and at least one processingunit including at least one processor operatively coupled to the imageprojecting unit and the sensor arrangements. The processing unit isconfigured for: analyzing one or more of the images captured by the atleast one shooter-side image sensor to detect projectile discharges inresponse to firing of the firearms by the shooters, and translatingdetections of projectile strikes on the virtual target, based on thedata collected by the at least one target-side sensor, and the detectedprojectile discharges into virtual actions in the virtual environment.Optionally, the at least one image projecting unit includes a pluralityof image projecting unit, and the at least one shooter-side sensorarrangement includes a plurality of shooter-side sensor arrangements,and the at least one target-side sensor arrangement includes a pluralityof target-side sensor arrangement, and the at least one processing unitincludes a plurality of processing units, and each of the processingunits is linked to a server via a wired or wireless network.

Optionally, a first one of the image projecting units, a first one ofthe shooter-side sensor arrangements, a first one of the target-sidesensor arrangements, and a first one of the processing units aredeployed together in a first geographic location, and a second one ofthe image projecting units, a second one of the shooter-side sensorarrangements, a second one of the target-side sensor arrangements, and asecond one of the processing units are deployed together in a secondgeographic location different from the first geographic location.

Optionally, the virtual environment further includes a plurality ofvirtual entities, each respective virtual entity representing arespective shooter, and the virtual actions include virtual firingactions performed by the virtual entity and virtual strikes on thevirtual target.

There is also provided according to an embodiment of the teachings ofthe present invention a system for jointly training a plurality ofshooters periodically firing a plurality of respective firearms. Thesystem comprises: a plurality of shooter-side image sensors including atleast a first and a second shooter-side image sensor, the firstshooter-side image sensor being deployed to capture images of a firstsubset of the shooters, and the second shooter-side image sensordeployed to capture images of a second subset of the shooters, the firstand second shooter-side image sensors being located in differentgeographic locations; a plurality of target-side sensors including atleast a first and a second target-side sensor, the first target-sidesensor deployed to collect data indicative of projectile strikes on atleast one target projected as part of a shared virtual environment on afirst background, the second target-side sensor deployed to collect dataindicative of projectile strikes on the target projected as part of theshared virtual environment on a second background, the first and secondtarget-side sensors collocated with the first and second shooter-sideimage sensors, respectively; a plurality of image projecting unitsincluding at least a first and a second image projecting unit, the firstimage projecting unit displaying the shared virtual environment on thefirst background, and the second image projecting unit displaying theshared virtual environment on the second background, and the sharedvirtual environment including a plurality of virtual entities, eachvirtual entity representing a respective shooter based on one or moreimages captured by the shooter-side image sensors; a plurality ofprocessing units including at least a first and a second processingunit, each of the processing units including at least one processorcoupled to a storage medium; and a server linked to the processing unitsover a wired or wireless network. The first processing unit isconfigured for analyzing one or more of the images captured by the firstshooter-side image sensor to detect projectiles discharges in responseto firing of the firearms by the first subset of shooter. The secondprocessing unit is configured for analyzing one or more of the imagescaptured by the second shooter-side image sensor to detect projectilesdischarges in response to firing of the firearms by the second subset ofshooter. The server is configured for co-processing detections ofprojectile strikes on the target, based on the data collected by thefirst and second target-side sensors and the detected projectile strikesto translate the projectile discharges and the projectile strikes intoshared virtual actions in the shared virtual environment.

Unless otherwise defined herein, all technical and/or scientific termsused herein have the same meaning as commonly understood by one ofordinary skill in the art to which the invention pertains. Althoughmethods and materials similar or equivalent to those described hereinmay be used in the practice or testing of embodiments of the invention,exemplary methods and/or materials are described below. In case ofconflict, the patent specification, including definitions, will control.In addition, the materials, methods, and examples are illustrative onlyand are not intended to be necessarily limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention are herein described, by wayof example only, with reference to the accompanying drawings. Withspecific reference to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

Attention is now directed to the drawings, where like reference numeralsor characters indicate corresponding or like components. In thedrawings:

FIG. 1 is a schematic illustration of a system, operative, according toembodiments of the present disclosure, for joint firearm training ofshooters, the system including a shooter-side sensor arrangement, atarget-side sensor arrangement, and a processing unit;

FIG. 2 is a block diagram of the processing unit of the system,according to embodiments of the present disclosure;

FIG. 3 is a block diagram of the shooter-side sensor arrangement,according to embodiments of the present disclosure;

FIG. 4 is a block diagram of the target-side sensor arrangement,according to embodiments of the present disclosure;

FIG. 5 is a block diagram of the shooter-side sensor arrangement,according to other embodiments of the present disclosure;

FIG. 6 is a block diagram of the target-side sensor arrangement,according to other embodiments of the present disclosure;

FIG. 7 is a flow diagram illustrating a process for joint firearmtraining of shooters, according to embodiments of the presentdisclosure;

FIG. 8 is a flow diagram illustrating a process for joint firearmtraining of shooters in which the target-side sensor arrangementincludes a target-side image sensor, according to embodiments of thepresent disclosure;

FIG. 9A is a schematic representation of a field-of-view (FOV)associated with a shooter-side image sensor of the shooter-side sensorarrangement, according to embodiments of the present disclosure;

FIG. 9B is a schematic representation of the FOV of FIG. 9A beingsub-divided into multiple regions, with a different shooter positionedin each region, according to embodiments of the present disclosure;

FIG. 10 is a schematic illustration of a system, similar to the systemof FIG. 1, that includes an image projecting unit that projects virtualshooter entities onto a background, according to embodiments of thepresent disclosure;

FIG. 11 is a schematic illustration of a system, operative according toembodiments of the present disclosure, operating over a network; and

FIG. 12 is a schematic front view illustrating a target area thatincludes multiple targets, according to embodiments of the presentdisclosure.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is directed to systems and methods for jointfirearm training of shooters using live fire projectiles.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

Embodiments of the present disclosure may be used to advantage inorganized weapons training facilities operated by, for example,military, law enforcement and/or security organizations, in which largenumbers of trainees (i.e., shooters) undergo firearm weapons training injoint training exercises. In such training facilities, multiple shootersmay fire weapons at a single shared target, or an array of targets maybe deployed with each shooter fire a weapon at a respective designatedtarget. Embodiments of the present disclosure may also be used toadvantage to allow collaborative training between multiple shooterslocated in different geographical locations. For example, a first subsetof shooters may be deployed in a first geographical location and firerespective weapons at a first set of one or more targets, and a secondsubset of shooters may be deployed in a second geographical locationsand fire respective weapons at a second set of one or more targets.

Referring now to the drawings, FIG. 1 illustrates a system, generallydesignated 100, for collaborative, i.e., joint firearm training ofshooter, according to an embodiment of the present disclosure. Thesystem 100 is deployable in various locations, but is preferablydeployed at a firearm training location, such as a shooting range.

A plurality of shooters each aim a designated firearm (i.e., weapon) ata target 126 and operate the designated firearm by firing the firearm todischarge a ballistic projectile (i.e., a live fire projectile, e.g.,bullet or round) with a goal of striking the target 126 with thedischarged ballistic projectile. In the non-limiting example deploymentof the system 100 illustrated in FIG. 1, three such shooters aredepicted, namely a first shooter 102, a second shooter 110, and a thirdshooter 118. The first shooter 102 operates a first firearm 104 todischarge a first projectile 106. The second shooter 110 operates asecond firearm 112 to discharge a second projectile 114. The thirdshooter 118 operates a third firearm 120 to discharge a third projectile122.

The target 126 is mounted to a background 128, which may be implementedas a rack, stand or holder for holding the target 126. As the target 126may be in principle smaller than the background 128 onto which it ismounted, the overall area which encompasses the target 126 and thebackground 128 is referred to herein as a target area 125.

The shooters 102, 110, 118 are positioned opposite (i.e., across from)the target 126 in spaced relation to each other, in a manner such thatthe shooters 102, 110, 118 do not interfere with each other when firingon the target 126 simultaneously. In certain operating conditions, theshooters 102, 110, 118 are positioned, for example in a line, such thatthey are approximately the same distance from the target 126.

Note that although FIG. 1 shows multiple shooters aiming respectivefirearms at a single common target (i.e., the target 126), the jointfirearm training methodologies of the present disclosure are alsoapplicable to situations in which the target area 125 covers a pluralityof targets (arranged, for example, in an array) and each individualshooter aims his respective firearm at a respective dedicated target.For example, FIG. 12 shows a front view of a target area 225 that coversthree targets, namely a first target 226 a, a second target 226, and athird target 226 c, deployed in an array. The boundary of the targetarea 225 is demarcated by dashed lines for clarity. In the example ofFIG. 12, the first shooter 102 may operate the first firearm 104 tostrike the first target 226 a in the target array, the second shooter110 may operate the second firearm 112 to strike the second target 226 bin the target array, and the third shooter 118 may operate the thirdfirearm 120 to strike the third target 226 c in the target array.

A first sensor arrangement 130, referred to hereinafter as ashooter-side sensor arrangement 130, is deployed in front of theshooters 102, 110, 118 so as to cover a coverage area in which theshooters 102, 110, 118 are positioned. The shooter-side sensorarrangement 130 includes at least one image sensor, namely a firstshooter-side image sensor 132. The image sensor 132 may be implementedas part of a camera system, and within the context of the presentdisclosure the term image sensor and camera are used interchangeably.The image sensor 132 has an associated field-of-view (FOV) that coversthe coverage area. The shooter-side sensor arrangement 130 is deployedsuch that the shooters 102, 110, 118 and the correspondingly operatedfirearms 104, 112, 120 are within the FOV of the image sensor 132 so asto enable image capture of the shooters 102, 110, 118 and thecorrespondingly operated firearms 104, 112, 120 by the shooter-sideimage sensor 132.

In certain embodiments, more than one shooter-side image sensor may bedeployed, as shown, for example, in the non-limiting example deploymentof the system 100 illustrated in FIG. 1 which shows a secondshooter-side image sensor 138. Two shooter-side image sensors may beused in order to estimate the distance between the shooter-side sensorarrangement 130 and the shooters 102, 110, 118, as will be discussed insubsequent sections of the present disclosure.

In addition, deployment of more than one shooter-side image sensor maybe used for redundant image capture of the shooters, in which two (ormore) shooter-side image sensors capture images of the same subset ofshooters. For example, the first shooter-side image sensor 132 may bedeployed to capture images of the shooters 102 and 110, and the secondshooter-side image sensor 138 may be deployed to capture images of theshooters 110 and 118, resulting in both shooter-side image sensors 132and 138 capturing images of the second shooter 110. Alternatively, forexample, the shooter-side image sensors 132 and 138 may both be deployedto capture images of all three shooters 102, 110, 118.

The use of more than one shooter-side image sensor may also be used toadvantage in situations in which the FOV of one shooter-side imagesensor is not wide enough to effectively capture images of all of theshooters and corresponding firearms. In such situations, theshooter-side image sensors may be deployed such that the correspondingFOVs allow image capture of different subsets of the shooters andcorresponding firearms. For example, the first shooter-side image sensor132 may be deployed with a FOV allowing image capture of the shooters102 and 110 and their corresponding firearms 104 and 112, while thesecond shooter-side image sensor 138 may be deployed with a FOV to allowimage capture of the third shooter 118 and the third firearm 120. Assuch, as the number of shooters firing on the same target grows,additional shooter-side image sensors can be deployed to ensure imagecapture of all of the shooters. As should be understood, the subsets ofshooters may be non-overlapping, or may be overlapping to allow formeasuring the position of each of the shooters and the distance of eachshooter from the shooter-side sensor arrangement 130 or for redundantimage capture, as described above.

The shooter-side image sensor 132 (or sensors 132 and 138) arepreferably implemented as infrared (IR) detection capable image sensorsthat operate at an image capture rate (i.e., frame rate) that is highenough, for example 90 frames per second (fps) or more, to supportdetection of projectile discharges (i.e., exit blasts from fired shots)from the firearms 104, 112, 120 in response to the shooters firing therespective firearms. One example of such an image sensor is the ICX424ALCCD sensor from Sony, available as part of the acA640-90gc camera systemfrom Basler.

The number of shooters for which a single shooter-side image sensor cansuccessfully capture images thereof, with high enough resolution tosupport detection of projectile discharges, is a function of the opticalparameters of the shooter-side image sensor. The shooter-side imagesensor 132, when implemented, for example, as the above-mentioned IRdetection capable image sensor, can support image capture of anywherebetween 1 and 10 shooters when positioned at an average distance in therange of 6-8.5 meters from each shooter. The distance (i.e., range)between the shooter-side sensor arrangement 130 and the shooter 102,110, 118 and the sensor arrangement 130 width of the FOV may be afunction of the optical parameters of the shooter-side image sensor 132and the number of non-overlapping image sensors such as the second imagesensor 138.

A second sensor arrangement 146, referred to hereinafter as atarget-side sensor arrangement 146, is deployed in front of the targetarea 125 to cover a coverage area in which the target area 125 ispositioned. The target-side sensor arrangement 146 includes at least onetarget-side projectile strikes detection sensor 148 deployed to collectdata indicative of projectile strikes on the target area 125 in order toallow detection of projectile strikes on the target area 125. In anon-limiting exemplary implementation, the projectile strikes detectionsensor 148 is implemented as an image sensor. Periodically throughoutremaining sections of the present disclosure, the projectile strikesdetection sensor 148 will be described within the context of theaforesaid non-limiting exemplary image sensor implementation,specifically when discussing sensor parameters that are applicable toimage sensor specific implementations, such as, for example, opticalparameters and field-of-view (FOV). It is noted, however, that theprojectile strikes detection sensor 148 may be implemented as any othertype of sensor capable of detecting projectile strikes on the targetarea 125, including, but not limited to, pressure sensors and acousticbased sensors, such as, for example Location of Miss and Hit (LOMAH)type sensors. In non-image sensor-based implementations of thetarget-side sensor 148, the target-side sensor 148 is configured todetect projectile strikes on the target area 125 based on the datacollected by the target-side sensor 148.

The target-side projectile strikes detection sensor 148, whenimplemented as an image sensor, has an associated FOV that covers thecoverage area. The target-side sensor arrangement 146 is deployed suchthat the target area 125 is within the FOV of the target-side imagesensor 146. The target-side projectile strikes detection sensor 148 ispreferably implemented as part of a camera system that operates at ahigh enough frame rate and resolution to support detection of projectilestrikes on the target area 125. The target-side projectile strikesdetection sensor 148 may be implemented using an image sensor of thesame type as the shooter-side image sensor 132. However, the target-sideimage sensor may operate at a lower frame rate than that of theshooter-side image sensor 132, for example 25 fps, and still supportdetection of projectile strikes on the target area 125 (as long as theresolution and optical parameters of the target-side image sensor enabledetection of projectile strikes on the target area 125).

In a non-limiting example deployment, the target-side sensor arrangement146 is deployed such that the distance between the target area 125 andthe target-side projectile strikes detection sensor 148 is in the rangeof 0.8-1.5 meters. The range between the target-side sensor arrangement146 and the target area 125 may be a function of the width of the FOV ofthe target-side projectile strikes detection sensor 148.

In embodiments in which each shooter aims his respective firearm at arespective dedicated target, a single target-side sensor 148 may bedeployed to collect data indicative of projectile strikes on each of thetargets. Alternatively, the target-side sensor arrangement 146 mayinclude a plurality of target-side projectile strikes detection sensors148, with each target-side projectile strikes detection sensor 148deployed to collect data indicative of projectile strikes on arespective one of the plurality of targets.

In operation, the shooter-side image sensor 132 (or sensors 132 and 138)captures a series of images of the shooters 102, 110, 118 and therespectively operated firearms 104, 112, 120, and the target-sideprojectile strikes detection sensor 148 captures a series of images ofthe target area 125. The target-side projectile strikes detection sensor148 and the shooter-side image sensor 132 are synchronized by aprocessing unit 156.

It is noted that the terms “series of images” and “sequence of images”may be used interchangeably throughout this document, and that theseterms carry with them an inherent temporal significance such thattemporal order is preserved. In other words, a first image in the seriesor sequence of images that appears prior to a second image in the seriesor sequence of images, implies that the first image was captured at atemporal instance prior to the second image.

With continued reference to FIG. 1, refer now to FIG. 2, a block diagramof the processing unit 156. The processing unit 156 includes a processor158 coupled to an internal or external storage medium 160 such as amemory or the like, and a clock 161. The external storage medium 160 maybe implemented as an external memory device connected to the processingunit 156 via a data cable or other physical interface connection, or maybe implemented as a network storage device or module, for example,hosted by a remote server (e.g., a cloud server). The clock 161 includestiming circuitry for synchronizing the sensor arrangements 130 and 146.The processing unit 156 is configured to apply image processing andcomputer vision algorithms to identify changes in a scene based onimages of the scene captured over an interval of time. In certainembodiments of the present disclosure, the processing unit 156 appliessuch algorithms to images of two separate scenes, namely a first scenein which the target area 125 is located, and a second scene in which theshooters 102, 110, 118 are located.

The processor 158 and the storage medium 160, although shown as a singlecomponent for representative purposes, may be multiple components (i.e.,multiple processors and/or multiple storage mediums). The processor 158can be implemented as any number of computer processors, including, butnot limited to, a microprocessor, microcontroller, an ASIC, a DSP, and astate machine. In certain non-limiting implementations, the processor158 is advantageously implemented as an image processor. All of suchprocessors include, or may be in communication with non-transitorycomputer readable media, such as, for example, the storage medium 160.Such non-transitory computer readable media store program code orinstructions sets that, when executed by the processor 158, cause theprocessor 158 to perform actions. Types of non-transitory computerreadable media include, but are not limited to, electronic, optical,magnetic, or other storage or transmission devices capable of providinga processor, such as the processor 158, with computer readableinstructions.

In certain embodiments, the processing unit 156 is configured toreceive, from the shooter-side sensor arrangement 130, the series ofimages captured by the shooter-side image sensor 132. The processingunit 156 analyzes the received series of images captured by theshooter-side image sensor 132 to detect projectile discharge events(referred to interchangeably as “projectile discharges”) from each ofthe firearms of the shooters in the respective FOV of the shooter-sideimage sensor 132. Each detected projectile discharge is made in responseto a shooter firing his respective firearm. For example, in anon-limiting implementation in which the first shooter-side image sensor132 is deployed to capture images of all three of the shooters 102, 110,118, the processing unit 156 is configured to detect the discharging ofthe projectiles 106, 114, 122, in response to the shooters 102, 110, 118firing the respective firearms 104, 112, 120, thereby yielding threeprojectile discharge events.

The processing unit 156 may analyze the received shooter-side images invarious ways. In a non-limiting implementation, the images captured bythe shooter-side image sensor 132 are IR images. In a preferred butnon-limiting exemplary implementation, the processing unit 156implements machine/computer vision techniques to identify flashes,corresponding to projectile discharges, from the barrel of the firearm.In another non-limiting exemplary implementation, the processing unit156 may detect projectile discharges via thermographic techniques, forexample by detecting the heat signature of the projectile as it leavesthe barrel of the firearm.

In another non-limiting implementation, which may be alternative to orin combination with the machine/computer vision techniques orthermographic implementation, individual images in the series of imagesare compared with one or more other images in the series of images toidentify changes between images, in order to identify the flashes comingfrom the barrel of the firearm corresponding to projectile discharges.

The processing unit 156 may further link an identified projectiledischarge with the firearm that discharged the projectile by identifyingeach of the firearms and/or shooters via various identification methods,as will be described in subsequent sections of the present disclosure.

The linking may be performed by determining which of the identifiedfirearms and/or shooters is closest in proximity to which of theidentified projectile discharges. The proximity may be evaluated on aper pixel level, for example by determining the differences in pixellocation (in the series of images) between image pixels indicative of aprojectile discharge and image pixels indicative of an identifiedfirearm and/or shooter.

In certain embodiments, for example non-imaging implementations of thetarget-side projectile strikes detection sensor 148, the processing unit156 is further configured to receive, from the target-side sensorarrangement 146, projectile strike events detected by the target-sideprojectile strikes detection sensor 148. The processing unit 156analyzes the received projectile strike events (referred to hereinafteras “projectile strikes”) on the target area 125. Each detectedprojectile strike is made in response to a projectile striking thetarget area 125. The detected projectile strikes correspond to theprojectiles discharged by the firearms in response to shooters firingtheir respective firearms. In embodiments in which the target-sideprojectile strike detection sensor 148 is implemented as an imagesensor, the processing unit 156 is configured to analyze a series ofimages of the target area 125 captured by the target-side projectilestrikes detection sensor 148.

The processing unit 156 may analyze the received target-side projectilestrike events in various ways. In a non-limiting implementation,individual images in the series of images are compared with one or moreother images in the series of images to identify changes between images,in order to identify projectile strikes on the target area 125.

The projectile strike identification also includes identification oflocation on the target area 125 at which the projectile strike occurred,and as such may also provide a projectile strike accuracy score ormetric. For example, the processing unit 156 may identify which portionof the target 126 a specific projectile struck in order to classify theprojectile strikes as hits or misses. For example, as illustrated inFIG. 1, the first shooter 102 fires the first firearm 104 such that theprojectile 106 strikes the target area 125 but misses the target 126.The second shooter 110 fires the second firearm 112 such that theprojectile 114 strikes the outer ring of the target 126. The thirdshooter 118 fires the third firearm 120 such that the projectile 122strikes the middle ring of the target 126. Accordingly, the processingunit 156 may assign the third shooter 118 with the highest accuracymetric or score, and the may assign the first shooter 102 with thelowest accuracy metric or score. As should be apparent, the processingunit 156 may aggregate or accumulate the projectile strikes to providean overall accuracy metric or score per shooter.

The image comparison methods, mentioned above within the context ofdetecting projectile strikes, may include pairwise comparisons of imagesto determine whether such images are identical. If such image pairs aredeemed to be identical, no detection of projectile strikes is made. Ifsuch image pairs are deemed to be non-identical, a detection of aprojectile strike is made. The term “identical” refers to images whichare determined to be closely matched by the processing unit 156, suchthat a change in the scene is not detected by the processing unit 156.The term “identical” is not intended to limit the functionality of theprocessing unit 156 to detecting changes to the scene only if thecorresponding pixels between two images have the same value.

With respect to the above described processes for detecting projectiledischarges and projectile strikes on the target area 125, the processingunit 156 is preferably configured to execute one or more imagecomparison algorithms, which utilize one or more computer vision and/orimage processing techniques. In one example, the processing unit 156 maybe configured to execute keypoint matching computer vision algorithms,which rely on picking points, referred to as “key points”, in the imagewhich contain more information than other points in the image. Anexample of keypoint matching is the scale-invariant feature transform(SIFT), which can detect and describe local features in images,described in U.S. Pat. No. 6,711,293.

In another example, the processing unit 156 may be configured to executehistogram image processing algorithms, which bin the colors and texturesof each captured image into histograms and compare the histograms todetermine a level of matching between compared images. A threshold maybe applied to the level of matching, such that levels of matching abovea certain threshold provide an indication that the compared images arenearly identical, and that levels of matching below the thresholdprovide an indication that the compared images are demonstrablydifferent.

In yet another example, the processing unit 156 may be configured toexecute keypoint decision tree computer vision algorithms, which relieson extracting points in the image which contain more information,similar to SIFT, and using a collection decision tree to classify theimage. An example of keypoint decision tree computer vision algorithmsis the features-from-accelerated-segment-test (FAST), the performance ofwhich can be improved with machine learning, as described in “MachineLearning for High-Speed Corner Detection” by E. Rosten and T. Drummond,Cambridge University, 2006.

As should be understood, results of such image comparison techniques maynot be perfectly accurate, resulting in false detections and/or misseddetections, due to artifacts such as noise in the captured images, anddue to computational complexity. However, the selected image comparisontechnique may be configured to operate within a certain tolerance valueto reduce the number of false detections and missed detections.

Although not show in the drawings, the processing unit 156 may be linkedto a display in order to visually display the projectile strikes on thetarget area 125 for each individual shooter (or cumulatively for allshooters). Alternatively, or in addition, the processing unit 156 may belinked with a centralized server 162 via a network 164 to allow eachshooter to access, via a computing device (e.g., smartphone, tablet,personal computer, laptop, etc.) linked to the network 164, personalizedshooting accuracy performance. The network 164 may be formed of one ormore wired or wireless networks, including, for example, the Internet,cellular networks, wide area, public, and local networks. Although notshown in the drawings, a communications module, including a networkinterface, may be coupled to the processing unit 156 to provide acommunication link between the processing unit 156 and the network 164.

The server 162 may be implemented as a remote server, such as, forexample, a cloud server or server system. The server includes acomputerized processor, such as, for example, a microprocessor, whichmay be configured to perform some or all of the image processingfunctionality previously described as attributed to the processing unit156. As such, in certain embodiments, all of the image processingfunctionality may be offloaded to the server 162. In other embodiments,the image processing functionality may be shared between the processingunit 156 and the server 162.

In order to properly assign the accuracy scores or metrics to theindividual shooters, the processing unit 156 first correlates theprojectile discharges (which are linked to individual shooters) withprojectile strikes. Before explaining the correlation process, attentionis first directed to FIGS. 3 and 4, which illustrate block diagrams ofthe shooter-side sensor arrangement 130 and the target-side sensorarrangement 146, respectively, according to an embodiment of the presentdisclosure.

The shooter-side sensor arrangement 130 includes the first shooter-sideimage sensor 132 (and optionally the second shooter-side image sensor138), and in certain embodiments includes a clock 145 and/or a distancemeasuring unit 144. The target-side sensor arrangement 146 includes thetarget-side projectile strikes detection sensor 148, and in certainembodiments includes a clock 155 and/or a distance measuring unit 154.The clocks 145 and 155 include timing circuitry which may be utilized toaid in synchronization between shooter-side sensor arrangement 130 andthe processing unit 156. The clocks 145 and 155 may also providetemporal information (e.g., timestamp information), to the processingunit 156, for each of the images captured by the image sensors 132, 138,148. In other embodiments, the processing unit 156 may apply timestampsto the data received from the sensor arrangements 130 and 146, therebyproviding temporal information for the detection events (i.e., theprojectile discharge events and the projectile strike events).

In embodiments in which the shooter-side sensor arrangement 130 includesa distance measuring unit 144, the distance measuring unit 144 isconfigured to measure (i.e., estimate) the distance between theshooter-side sensor arrangement 130 and each of the shooters 102, 110,120. The distance measuring unit 144 may be implemented, for example, asa laser rangefinder that emits laser pulses for reflection off of atarget (i.e., the shooters) and calculates distance based on the timedifference between the pulse emission and receipt of the reflectedpulse.

In certain embodiments, the distance measuring unit 144 may be absentfrom the shooter-side sensor arrangement 130, and the distance betweenthe shooter-side sensor arrangement 130 and each of the shooters 102,110, 120 may be calculated using principles of triangulation (i.e.,stereoscopic imaging) based on images captured by two shooter-side imagesensors (e.g., the first and second shooter-side image sensors 132 and138) that are synchronized with each other. Alternatively, a singleshooter-side image sensor, implemented as part of a stereo vision camerasystem, such as the Karmin2 stereo vision camera available from SODAVISION, may be used to measure the distance between the shooter-sidesensor arrangement 130 and each of the shooters 102, 110, 120.

In embodiments in which the target-side sensor arrangement 146 includesa distance measuring unit 154, the distance measuring unit 154 isconfigured to measure the distance between the target-side sensorarrangement 146 and the target area 125. The distance measuring unit 154may be implemented, for example, as a laser rangefinder. In certainembodiments, the distance measuring unit 154 may be absent from thetarget-side sensor arrangement 146, and the distance between thetarget-side sensor arrangement 146 and the target area 125 may becalculated (i.e., estimated) by applying image processing techniques,performed by the processing unit 156, to images of a visual markerattached to the target area 125. The visual marker may be implemented,for example, as a visual mark of a predefined size. The number of pixelsdedicated to the portion of the captured image that includes the visualmark can be used as an indication of the distance between thetarget-side sensor arrangement 146 and the target area 125. For example,if the target-side sensor arrangement 146 is positioned relatively closeto the visual mark, a relatively large number of pixels will bededicated to the bar code portion of the captured image. Similarly, ifthe target-side sensor arrangement 146 is positioned relatively far fromthe visual mark, a relatively small number of pixels will be dedicatedto the bar code portion of the captured image. As a result, a mappingbetween the pixel density of portions of the captured image and thedistance to the object being imaged can be generated by the processingunit 156, based on the visual mark size.

Note that the system 100 may determine the aforementioned respectivedistances between the sensor arrangements 130 and 146 and the shootersand the target area in various ways. As discussed above, in certainembodiments, the distance is determined (i.e., estimated) via thedistance measuring units 144 and 154. In other embodiments, an operatorof the system 100, which may be, for example, a manager of the shootingrange in which the system 100 is deployed, or one or more of theshooters 102, 110, 118, may manually input the aforementioned distancesto the processing unit 156. In such embodiments, manual input to theprocessing unit 156 may be effectuated via user interface (e.g., agraphical user interface) executed by a computer processor on a computersystem linked to the processing unit 156. In such embodiments, theprocessing unit 156 may be deployed as part of the computer system thatexecutes the user interface.

In certain embodiments, the sensor arrangements 130 and 146 areapproximately collocated. The two distances (i.e., between theshooter-side sensor arrangement 130 and the shooters, and between thetarget-side sensor arrangement 146 and the target area 125) are summedby the processing unit 156 to calculate (i.e., estimate) the distancebetween the target area 125 and shooters 102, 110, 118. As mentionedabove, the typical distance between the shooter-side sensor arrangement130 and the shooters 102, 110, 118 is preferably in the range of 6-8.5meters, and the distance between the target-side sensor arrangement 146and the target area 125 is preferably in the range of 0.8-1.5 meters.Accordingly, in a non-limiting deployment of the system 100, thedistance between the shooters 102, 110, 118 and the target area 125 isin the range of 6.8-10 meters.

In other embodiments, the sensor arrangements 130 and 146 are spacedapart from each other at a pre-defined distance. Spacing the sensorarrangements 130 and 146 apart at a pre-defined distance may supportlong-range shooting capabilities, in which the distance between theshooters 102, 110, 118 and the target area 125 may be greater than 10meters (for example several tens of meters and up two several hundredmeters). In such an embodiment, the distance between the shooter-sidesensor arrangement 130 and the shooters, between the target-side sensorarrangement 146 and the target area 125, and the pre-defined distancebetween the sensor arrangements 130 and 146 are summed by the processingunit 156 to calculate the distance between the target area 125 andshooters 102, 110, 118.

Based on the calculated distance between the target area 125 andshooters 102, 110, 118, and the average speed of a dischargedprojectile, the processing unit 156 determines an expected time offlight (ToF), defined as the time a discharged projectile will take tostrike the target area 125, for each firearm. The processing unit 156may store the expected ToFs for each firearm in a memory (e.g., thestorage medium 160) or in a database as a data record with header orfile information indicating to which firearm (i.e., shooter) eachexpected ToF corresponds.

It is noted that the range between the object (e.g., shooters or target)to be imaged and the sensor arrangements 130 and 146 may be increased invarious ways. For example, higher resolution image sensors, or imagesensors with larger optics (e.g., lenses) and decreased FOV, may be usedto increase the range. Alternatively, multiple shooter-side imagesensors with non-overlapping FOVs may be deployed to increase theoperational range between the shooters and the shooter-side sensorarrangement 130.

In operation, for each detected projectile strike, the processing unit156 evaluates the temporal information (i.e., timestamp) associated withthe projectile strike. The processing unit 156 also evaluates thetemporal information associated with recently detected projectiledischarges. The processing unit 156 then compares the temporalinformation associated with the projectile strike with the temporalinformation associated with recently detected projectile discharges. Thecomparison may be performed, for example, by taking the pairwisedifferences between the temporal information associated with recentlydetected projectile discharges and the temporal information associatedwith the projectile strike to form estimated ToFs. The estimated ToFsare then compared with the expected ToFs to identify a closest matchbetween estimated ToFs and expected ToFs. The comparison may beperformed by taking the pairwise differences between the estimated ToFsand the expected ToFs, and then identifying the estimated ToF andexpected ToF pair that yields the minimum (i.e., smallest) difference.

Since the processing unit 156 provides synchronization between theevents detected in response to the data received from the sensorarrangements 130 and 146, which in certain embodiments is provided viasynchronization of the clocks 145, 155, 161, the processing unit 156 isable to perform the ToF calculations with relatively high accuracy,preferably to within several micro seconds. Furthermore, by identifyingthe estimated ToF and expected ToF pair, the processing unit 156 is ableto retrieve the stored information indicative of to which firearm (i.e.,shooter) is associated with the expected ToF, thereby attributing thedetected projectile strike to the shooter operating the firearmassociated with the expected ToF of the identified estimated ToF andexpected ToF pair. As such, the processing unit 156 is able to identify,for each detected projectile strike on the target area 125, thecorrespondingly fired firearm that caused the detected projectilestrike.

The processing unit 156 may also be configured to provide target missinformation for projectile discharges that failed to hit the target 126or the target area 125. To do so, the processing unit 156 may evaluatetemporal information associated with each detected projectile discharge.The processing unit 156 also evaluates the temporal informationassociated with recently detected projectile strikes. The processingunit 156 then compares the temporal information associated with theprojectile discharge with the temporal information associated withrecently detected projectile strikes. The comparison may be performed,for example, by taking the differences between the temporal information,similar to as described above, to form estimated ToFs. Pairwisedifferences between the estimated ToFs and the expected ToFs may then beperformed. The estimated ToF and expected ToF pair that yields theminimum difference but is greater than a threshold value is attributedto the firearm (i.e., shooter) associated with the expected ToF as atarget miss.

Although embodiments of the system 100 as described thus far havepertained to a processing unit 156 (and/or a server 162) performingimage processing techniques to detect projectile discharges andprojectile strikes, other embodiments are possible in which one or bothof the sensor arrangements 130 and 146 include one or more processorshaving image processing capabilities. FIGS. 5 and 6 illustrate blockdiagrams of the shooter-side sensor arrangement 130 and the target-sidesensor arrangement 146, respectively, according to such an embodiment.

As shown in FIG. 5, the shooter-side sensor arrangement 130 includes aprocessor 134 coupled to the first shooter-side image sensor 132 and astorage medium 136, and a processor 140 coupled to the secondshooter-side image sensor 138 and a storage medium 142.

As shown in FIG. 6, the target-side sensor arrangement 146 includes aprocessor 150 coupled to the target-side projectile strikes detectionsensor 148 and a storage medium 152.

Each of the processors 134, 140, 150 is generally similar to theprocessor 158, and should be understood by analogy thereto. Likewise,each of the storage mediums 136, 142, 152 is generally similar to thestorage medium 160, and should be understood by analogy thereto.

In the embodiment of the system 100 described with reference to FIGS. 5and 6, the shooter-side sensor arrangement 130 is configured to performimage processing techniques (via the processors 134 and 140) to detectprojectile discharges, and the target-side sensor arrangement 146 isconfigured to perform image processing techniques (via the processor150) to detect projectile strikes. In such an embodiment, the projectiledischarge and projectile strike information may be sent, by theshooter-side sensor arrangement 130 and the target-side sensorarrangement 146, respectively, to the processing unit 156. Theprocessing unit 156 may then perform the correlations, as describedabove, to identify, for each detected projectile strike on the targetarea 125, the correspondingly fired firearm that caused the detectedprojectile strike, and to identify target misses.

Attention is now directed to FIG. 7 which shows a flow diagram detailinga process 700 in accordance with embodiments of the present disclosure.The process 700 includes steps for jointly training a plurality ofshooters operating respective firearms. Reference is also made to someof the elements shown in FIGS. 1-6. The process 700 includescomputerized sub-processes performed by the system 100 and relatedcomponents, such as the processing unit 156, and in certain embodimentsthe processors 158 and/or 134 and/or 140 and/or 150 and/or the server162.

The process 700 begins at block 702, where the shooter-side image sensor132 (or sensors 132 and 138) capture series of images of the shootersand the target area, respectively.

The process 700 then moves to block 704, where projectile discharges aredetected based on image data in the series of images captured by theshooter-side image sensor 132 (or sensors 132 and 138).

The process then moves to block 706, where the shooters that fired therespective firearms that triggered the respective detected projectiledischarges are uniquely identified. Note that blocks 704 and 706 may beperformed in parallel or in reverse order than the order shown in FIG.7.

The process 700 then moves to block 708, where projectile strikes on thetarget area 125 are detected based on data collected by the target-sideprojectile strikes detection sensor 148.

As discussed above, in certain embodiments, the detection of theprojectile discharges (in block 704) and the unique identification ofshooters (in block 706) may be performed by the processing unit 156.

In other embodiments, the detection of the projectile discharges (inblock 704) and/or the unique identification of shooters (in block 706)may be performed by a processor 134 of the shooter-side sensorarrangement 130 that is coupled to the shooter-side image sensor 132 (orprocessors 134 and 140 respectively coupled to the shooter-side imagesensors 132 and 138).

In yet other embodiments, the detection of the projectile discharges (inblock 704) and the unique identification of shooters (in block 706) maybe performed by the server 162 alone or in combination with theprocessing unit 156.

It is noted that the processing system that performs the aforementioneddetections performs such detections contemporaneously and in real-time.

The process 700 then moves to block 710, where the detected projectiledischarges and the detected projectile strikes are correlated to createa link between projectile discharges and projectile strikes.Specifically, the correlation performed in block 710 providesidentification, for each detected projectile strike on the target area125, of the correspondingly fired firearm (and shooter that fired thefirearm) that caused the detected projectile strike. The correlationperformed in block 710 further provides identification of projectiledischarges that fail to correspond to any projectile strike, therebyclassifying such projectile discharges as misses.

Attention is now directed to FIG. 8 which shows a flow diagram detailinga process 800 in accordance with embodiments of the present disclosure.The process 800 includes steps for jointly training a plurality ofshooters operating respective firearms when the target-side sensor 148is implemented as an image sensor.

Reference is also made to some of the elements shown in FIGS. 1-6. Theprocess 800 includes computerized sub-processes performed by the system100 and related components, such as the processing unit 156, and incertain embodiments the processors 158 and/or 134 and/or 140 and/or 150and/or the server 162.

The process 800 begins at block 802, where the shooter-side image sensor132 (or sensors 132 and 138) and the target-side projectile strikesdetection sensor 148 capture series of images of the shooters and thetarget area, respectively.

The process 800 then moves to block 804, where projectile discharges aredetected based on image data in the series of images captured by theshooter-side image sensor 132 (or sensors 132 and 138).

The process then moves to block 806, where the shooters that fired therespective firearms that triggered the respective detected projectiledischarges are uniquely identified. Note that blocks 804 and 806 may beperformed in parallel or in reverse order than the order shown in FIG.8.

The process 800 then moves to block 808, where projectile strikes on thetarget area 125 are detected based on image data in the series of imagescaptured by the target-side image sensor 148.

As discussed above, in certain embodiments, the detection of theprojectile discharges (in block 804), the unique identification ofshooters (in block 806), and the detection of the projectile strikes (inblock 806) may be performed by the processing unit 156.

In other embodiments, the detection of the projectile discharges (inblock 804) and/or the unique identification of shooters (in block 806)may be performed by a processor 134 of the shooter-side sensorarrangement 130 that is coupled to the shooter-side image sensor 132 (orprocessors 134 and 140 respectively coupled to the shooter-side imagesensors 132 and 138). In such embodiments, the detection of theprojectile strikes (in block 806) may be performed by a processor 150 ofthe target-side sensor arrangement 146 that is coupled to thetarget-side image sensor 148. The two processors 134 and 150 may besynchronized via the clocks 145 and 155.

In yet other embodiments, the detection of the projectile discharges (inblock 804), the unique identification of shooters (in block 706), andthe detection of the projectile strikes (in block 806) may be performedby the server 162 alone or in combination with the processing unit 156.

It is noted that the processing system that performs the aforementioneddetections performs such detections contemporaneously and in real-time.

The process 800 then moves to block 810, wherein the detected projectiledischarges and the detected projectile strikes are correlated to linkprojectile discharges with projectile strikes. Specifically, thecorrelation performed in block 810 provides identification, for eachdetected projectile strike on the target area 125, of thecorrespondingly fired firearm that caused the detected projectilestrike. The correlation performed in block 708 further providesidentification of projectile discharges that fail to correspond to anyprojectile strike, thereby classifying such projectile discharges asmisses.

As a result of the correlation performed in block 708 of the process 700and block 810 of the process 800, the system 100, and more specificallythe processing unit 156 and/or the server 162 may provide shooterspecific performance data and statistical data, for each individualshooter. The performance data may include, but is not limited to, thetotal number of projectiles fired, the average rate between projectiledischarges, the projectile strikes cluster on the target 126, theaverage hit point (the average point in the height and width coordinatesof a group of projectile strikes) and the average range of the shooterfrom the target area 125. The statistical data may include, but is notlimited to, strike/miss and accuracy data (e.g., based on the accuracyscores or metrics). The strike/miss data may include data indicating thepercentage of fired projectiles that struck the target 126, and dataindicating the percentage of fired projectiles that missed the target126 or the target area 125. The accuracy data may include averageaccuracy and the average strike position on the target 126.

As mentioned in previous sections of the present disclosure, theprocessing unit 156 may link each identified projectile discharge withthe firearm that discharged the projectile by identifying each of thefirearms and/or shooters via various identification methods. Theidentification of shooters and/or firearms, by components of the system100, may be performed using various machine learning and/or computervision algorithms and techniques. In certain embodiments, one or morevisual parameters associated with each of the shooters and/or firearmsare evaluated. The following paragraphs describe several exemplarymethods for identifying firearms and/or shooters, according toembodiments of the present disclosure.

In certain embodiments, the processing unit 156 is configured to analyzethe images captured by the shooter-side image sensor 132 (or imagesensors 132 and 138) using facial recognition techniques to identifyindividual shooters. In such embodiments, each of the shooters mayprovide a baseline facial image (e.g., digital image captured by acamera system) to the system 100, which may be stored in a memory of thesystem 100, for example the storage medium 160. The processing unit 156may extract landmark facial features (e.g., nose, eyes, cheekbones,lips, etc.) from the baseline facial image. The processing unit 156 maythen analyze the shape, position and size of the extracted facialfeatures. In operation, the processing unit 156 identifies facialfeatures in the images captured by the shooter-side image sensor 132 (orimage sensors 132 and 138) by searching through the captured images forimages with matching features to those extracted from the baselineimage.

In another embodiment, computer vision techniques are used to identifyshooters based on markers attached to the bodies of the shooters or thefirearms operated by the shooters. As shown in FIG. 1, a first marker108 is attached to a headpiece worn by the first shooter 102, a secondmarker 116 is attached to a headpiece worn by the second shooter 110,and a third marker 124 is attached to a headpiece worn by the thirdshooter 118.

In a non-limiting implementation, the markers 108, 116, 124 arecolor-coded markers, with each shooter/firearm having a uniquelydecipherable color. In the non-limiting example deployment of the system100 illustrated in FIG. 1 with three shooters, the first shooter 102 mayhave a red marker attached to his body or firearm 104, the secondshooter 110 may have a green marker attached to his body or firearm 112,and the third shooter 118 may have a blue marker attached to his body orfirearm 120. The marker colors may be provided to the processing unit156 prior to operation of the system 100. In operation, the processingunit 156 identifies the color-coded markers in the images captured bythe shooter-side image sensor 132 (or image sensors 132 and 138) whichenables identification of the individual shooters and/or firearms.

In another non-limiting implementation, the marker may be implemented asan information-bearing object, such as, for example, a bar code, thatcarries identification data. The bar code may store encoded informationthat includes the name and other identifiable characteristics of theshooter to which the bar code is attached. In operation, the processingunit 156 searches for bar codes in the images captured by theshooter-side image sensor 132 (or image sensors 132 and 138), and uponfinding such a bar code, decodes the information stored in the bar code,thereby identifying the shooter (or firearm) to which the bar code isattached.

In another embodiment, the processing unit 156 may be configured toidentify individual shooters according to geographic position of eachshooter within the FOV of the shooter-side image sensor 132 (or imagesensors 132 and 138. In such embodiments, the FOV of the shooter-sideimage sensor 132 (or image sensors 132 and 138 may be sub-divided intonon-overlapping sub-regions (i.e., sub-coverage areas), with eachshooter positioned in a different sub-region.

FIG. 9A shows a schematic representation of the FOV 168 of the firstshooter-side image sensor 132. FIG. 9B shows a schematic representationof the sub-division of the FOV 168 into three sub-regions, namely afirst sub-region 170, a second sub-region 172, and a third sub-region174. The first shooter 102 is positioned in the first sub-region 170,the second shooter 110 is positioned in the second sub-region 172, andthe third shooter 118 is positioned in the third sub-region 174. Thesub-division of the FOV 168 may be pre-determined (i.e., prior tooperation of the system 100 to perform the joint training disclosedherein). Likewise, the requisite position of each of the shooters, inthe respective sub-regions of the FOV may be pre-assigned and providedto the processing unit 156. In operation, the processing unit 156analyzes the images captured by the shooter-side image sensor 132 toidentify the shooters according to the pre-defined position in the FOVsub-regions 170, 172, 174.

Although embodiments of the system 100 as described thus far havepertained to detecting projectile discharges and projectile strikes, andcorrelating the detected discharges and strikes to link the dischargeswith strikes, embodiments of the present disclosure may also be used toadvantage when applied to collaborative and interactive virtual trainingscenarios which simulate real-life combat or combat-type situationsand/or firearms training and/or competitions, such as in a virtualfiring range.

Referring now to FIG. 10, there is shown a system, generally designated200, for performing joint (i.e., collaborative) firearm training ofshooters, according to an embodiment of the present disclosure. Thesystem 200 is generally similar to the system 100, except that thesystem 200 includes an image projecting unit 166 linked to theprocessing unit 156. The image projecting unit 166 is configured forprojecting images of a virtual training environment 176 on thebackground 128. In such embodiments, the background 128 may beimplemented as a projection screen. The image projecting unit 166 may beimplemented in various ways, including, for example, as a micro shortthrow projector, such as the LG PF1000UW Portable Ultra Short ThrowProjector or Sony LSPX-P1 Portable Ultra Short Throw Projector.

As an example of such a virtual training environment, the imageprojecting unit 166 may project a video or animated image of an armedhostage taker holding a hostage. In such a scenario, the hostage takeris treated by the system 200 as the target 126. As such, the position ofthe target 126 may change dynamically as the image of the hostage takermoves during progression of the training environment.

In response to a detected projectile strike or miss on the definedtarget (e.g., the hostage taker or other target object projected by theimage projecting unit 166), the processing unit 156 may actuate theimage projecting unit 166 to change the projected image. For example, ifthe image projecting unit 166 projects an image of a hostage takerholding a hostage, and one of the shooters discharges a projectile thatmisses the target (i.e., the hostage taker), the processing unit 156 mayactuate the image projecting unit 166 to change the projected image todisplay the hostage taker attacking the hostage.

As should be apparent, the above description of the hostage scenario isexemplary only, and is intended to help illustrate the functionality ofthe system 200.

In addition to the above, the image projecting unit 166 is configured toproject images of virtual entities 202, 210, 218 corresponding to therespective shooters 102, 110, 118. The virtual entities may be avatarsof the respective shooters, or may be images of the actual shooters. Insuch embodiments, the shooter-side image sensor 132 and the processingunit 156 cooperate to perform motion-capture functionality, such thatactions performed by the shooters 102, 110, 118 in the real-world (e.g.,firing of firearms, arm movement, head movement, pivoting, walking,etc.) are translated into virtual actions performed by the correspondingvirtual entities 202, 210, 218 in the virtual training environment 176.The processing unit 156 is also configured to translate detectedprojectile discharges in the real-world into virtual projectiledischarges in the virtual training environment 176. Furthermore, theprocessing unit 156 is configured to translate detected projectilestrikes in the real-world into virtual projectile strikes in the virtualtraining environment 176.

As discussed above, the processing unit 156 of the presently disclosedembodiments may be linked to a server 162 via a network 164. In additionto providing capabilities for allowing the shooters to accesspersonalized performance data and statistical data, and for offloadingsome or all of the image processing functionality from the processingunit 156, the server 162 may also provide connectivity between multipleshooters operating in different geographic locations.

FIG. 11 shows a networked system, generally designated 200′, in whichmultiple subsystems are deployed and interconnected with the server 162via the network 164. The system 200′ includes at least one subsystemgenerally similar to the system 200 of FIG. 10. As shown in FIG. 11, thesystem 200′ depicts a first subsystem, generally designated 200 a, andsecond subsystem, generally designated 200 b. Each of the subsystems 200a and 200 b are generally similar to the system 200. In general terms,the system 200′ can include up to N subsystems, in which N can take onany positive integer value. In implementations in which N=1, the system200′ is equivalent to the system 200 illustrated in FIG. 10.

The subsystems 200 a and 200 b are deployed in different respectivegeographic locations. Specifically, the sensor arrangements 130 a and146 a, the processing unit 156 a, and the image projecting unit 166 aare collocated in a first geographic location. Likewise, the sensorarrangements 130 b and 146 b, the processing unit 156 b, and the imageprojecting unit 166 b are collocated in a second geographic locationthat is different from the first geographic location.

The networked environment, that provides interconnection between thesubsystems 200 a and 200 b and the server 162, provides a joint trainingplatform in which multiple shooters, based in different geographiclocations, can participate in a shared virtual training environment.

In the non-limiting example deployment of the system 200′ illustrated inFIG. 10, a total of six shooters utilize the system 200′, with a firstsubset of shooters (i.e., the shooters 102 a, 110 a, 118 a) deployed touse the first subsystem 200 a in the first geographic location, and asecond subset of shooters (i.e., the shooters 102 b, 110 b, 118 b)deployed to use the second subsystem 200 b in the second geographiclocation.

In operation, the shooter-side image sensor of the shooter-side sensorarrangement 130 a captures images of the first plurality of shooters 102a, 110 a, 118 a, and the shooter-side image sensor of the shooter-sidesensor arrangement 130 b captures images of the second plurality ofshooters 102 b, 110 b, 118 b. Similarly, the target-side sensor of thetarget-side sensor arrangement 146 a detects projectile strikes on atarget 126 a that is projected as part of the shared virtual environmenton the background 128 a by the image projecting unit 166 a. Thetarget-side sensor of the target-side sensor arrangement 146 b detectsprojectile strikes on a target 126 b that is projected as part of theshared virtual environment on the background 128 b by the imageprojecting unit 166 b.

In addition to projecting the targets 126 a and 126 b, the imageprojecting units 166 a and 166 b are also configured to project theshared virtual environment on both backgrounds 128 a and 128 b. Theshared virtual environment includes images of virtual entitiescorresponding to all of the shooters (i.e., the shooters 102 a, 102 b,110 a, 110 b, 118 a, 118 b).

The processing units 156 a and 156 b are configured to perform localprojectile discharge and projectile strike detection. In other words,the processing unit 156 a is configured to analyze the images capturedby the shooter-side image sensor of the shooter-side sensor arrangement130 a, and receive projectile strike detections from the target-sideprojectile strikes detection sensor of the target-side sensorarrangement 146 a. In embodiments in which the target-side projectilestrikes detection sensor of the target-side sensor arrangement 146 a isimplemented as an image sensor, the processing unit 156 a is configuredto analyze the images captured by the target-side projectile strikesdetection sensor of the target-side sensor arrangement 146 a to detectprojectile strikes on the target 126 a. Similarly, the processing unit156 b is configured to analyze the images captured by the shooter-sideimage sensor of the shooter-side sensor arrangement 130 b, and receiveprojectile strike detections from the target-side projectile strikesdetection sensor of the target-side sensor arrangement 146 b. Inembodiments in which the target-side projectile strikes detection sensorof the target-side sensor arrangement 146 b is implemented as an imagesensor, the processing unit 156 b is configured to analyze the imagescaptured by the target-side projectile strikes detection sensor of thetarget-side sensor arrangement 146 b to detect projectile strikes on thetarget 126 b.

The server 162 is configured to receive the detected projectiledischarges and the detected projectile strikes from the processing units156 a and 156 b, and is configured to co-process the received detectedprojectile discharges and the detected projectile strikes to translatethe projectile discharges and the projectile strikes into shared virtualactions in the shared virtual environment. In certain embodiments, theserver 162 may also co-process the aforementioned detected projectiledischarges and the detected projectile strikes together with sensor data(e.g., image data in the form of captured images, acoustic sensor data,etc.) received from the shooter-side image sensors (of the shooter-sidesensor arrangements 130 a and 130 b) and target-side sensors (of thetarget-side sensor arrangements 146 a and 146 b) to translate theprojectile discharges and the projectile strikes into shared virtualactions in the shared virtual environment.

It is noted that although the embodiments of the present disclosuredescribed above have in many instances been provided within the contextof three shooters firing on a single target or three dedicated targets,such context was provided in order to better describe and illustrate theembodiments of the present disclosure. The embodiments of the presentdisclosure should not be limited to a specific number of shooters and/ortargets.

Implementation of the system and/or method of embodiments of theinvention can involve performing or completing selected tasks manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of embodiments of the method and/or systemof the invention, several selected tasks could be implemented byhardware, by software or by firmware or by a combination thereof usingan operating system.

For example, hardware for performing selected tasks according toembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to embodiments of theinvention could be implemented as a plurality of software instructionsbeing executed by a computer using any suitable operating system. Asdiscussed above, the data management application may be implemented as aplurality of software instructions or computer readable program codeexecuted on one or more processors of a mobile communication device. Assuch, in an exemplary embodiment of the invention, one or more tasksaccording to exemplary embodiments of method and/or system as describedherein are performed by a data processor, such as a computing platformfor executing a plurality of instructions. Optionally, the dataprocessor includes a volatile memory for storing instructions and/ordata and/or a non-volatile storage, for example, non-transitory storagemedia such as a magnetic hard-disk and/or removable media, for storinginstructions and/or data. Optionally, a network connection is providedas well. A display and/or a user input device such as a keyboard ormouse are optionally provided as well.

For example, any combination of one or more non-transitory computerreadable (storage) medium(s) may be utilized in accordance with theabove-listed embodiments of the present invention. The non-transitorycomputer readable (storage) medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

The block diagrams in the drawings illustrate the architecture,functionality, and operation of possible implementations of systems,devices, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

As used herein, the singular form, “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise.

The word “exemplary” is used herein to mean “serving as an example,instance or illustration”. Any embodiment described as “exemplary” isnot necessarily to be construed as preferred or advantageous over otherembodiments and/or to exclude the incorporation of features from otherembodiments.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

The processes (methods) and systems, including components thereof,herein have been described with exemplary reference to specific hardwareand software. The processes (methods) have been described as exemplary,whereby specific steps and their order can be omitted and/or changed bypersons of ordinary skill in the art to reduce these embodiments topractice without undue experimentation. The processes (methods) andsystems have been described in a manner sufficient to enable persons ofordinary skill in the art to readily adapt other hardware and softwareas may be needed to reduce any of the embodiments to practice withoutundue experimentation and using conventional techniques.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

What is claimed is:
 1. A system for jointly training a plurality ofshooters periodically firing a plurality of respective firearms, thesystem comprising: at least one shooter-side sensor arrangementincluding at least one shooter-side image sensor configured to captureimages of the shooters; at least one target-side sensor arrangementincluding at least one target-side sensor configured to collect dataindicative of projectile strikes on a virtual target; at least one imageprojecting unit configured to display a virtual environment on abackground, the virtual environment including the virtual target; and atleast one processing unit including at least one processor operativelycoupled to the image projecting unit and the sensor arrangements, atleast one of the target-side sensor arrangement and the processing unitconfigured to detect projectile strikes on the virtual target based onthe data collected by the at least one target-side sensor, and theprocessing unit configured to: analyze one or more of the imagescaptured by the at least one shooter-side image sensor to detectprojectile discharges in response to firing of the firearms by theshooters, and translate detections of projectile strikes on the virtualtarget and the detected projectile discharges into virtual actions inthe virtual environment.
 2. The system of claim 1, wherein the at leastone image projecting unit includes a plurality of image projectingunits, and wherein the at least one shooter-side sensor arrangementincludes a plurality of shooter-side sensor arrangements, and whereinthe at least one target-side sensor arrangement includes a plurality oftarget-side sensor arrangement, and wherein the at least one processingunit includes a plurality of processing units, and wherein each of theprocessing units is linked to a server via a wired or wireless network.3. The system of claim 2, wherein a first one of the image projectingunits, a first one of the shooter-side sensor arrangements, a first oneof the target-side sensor arrangements, and a first one of theprocessing units are deployed together in a first geographic location,and wherein a second one of the image projecting units, a second one ofthe shooter-side sensor arrangements, a second one of the target-sidesensor arrangements, and a second one of the processing units aredeployed together in a second geographic location different from thefirst geographic location.
 4. The system of claim 2, wherein the virtualenvironment further includes a plurality of virtual entities, eachvirtual entity representing a respective shooter, and wherein thevirtual actions include virtual firing actions performed by the virtualentity and virtual strikes on the virtual target.
 5. A system forjointly training a plurality of shooters periodically firing a pluralityof respective firearms, the system comprising: a plurality ofshooter-side image sensors including at least a first and a secondshooter-side image sensor, the first shooter-side image sensorconfigured to capture images of a first subset of the shooters, and thesecond shooter-side image sensor configured to capture images of asecond subset of the shooters, the first and second shooter-side imagesensors being located in different geographic locations; a plurality oftarget-side sensors including at least a first and a second target-sidesensor, the first target-side sensor configured to collect dataindicative of projectile strikes on at least one target projected aspart of a shared virtual environment on a first background, the secondtarget-side sensor configured to collect data indicative of projectilestrikes on at least one target projected as part of the shared virtualenvironment on a second background, the first and second target-sidesensors collocated with the first and second shooter-side image sensors,respectively; a plurality of image projecting units including at least afirst and a second image projecting unit, the first image projectingunit configured to display the shared virtual environment on the firstbackground, and the second image projecting unit configured to displaythe shared virtual environment on the second background, and the sharedvirtual environment including a plurality of virtual entities, eachvirtual entity representing a respective shooter based on one or moreimages captured by the shooter-side image sensors; a plurality ofprocessing units including at least a first and a second processingunit, each of the processing units including at least one processorcoupled to a storage medium, at least one of the first target-sidesensor arrangement and the first processing unit configured to detectprojectile strikes on the at least one target projected as part of theshared virtual environment on the first background based on the datacollected by the first target-side sensor, at least one of the secondtarget-side sensor arrangement and the second processing unit configuredto detect projectile strikes on the at least one target projected aspart of the shared virtual environment on the second background based onthe data collected by the second target-side sensor, the firstprocessing unit configured to analyze one or more of the images capturedby the first shooter-side image sensor to detect projectiles dischargesin response to firing of the firearms by the first subset of shooters,and the second processing unit configured to analyze one or more of theimages captured by the second shooter-side image sensor to detectprojectiles discharges in response to firing of the firearms by thesecond subset of shooter; and a server linked to at least the first andsecond processing units over a wired or wireless network and configuredto process the detected projectile discharges and the detections ofprojectile strikes on the at least one target projected as part of theshared virtual environment on the first and second backgrounds totranslate the projectile discharges and the projectile strikes intoshared virtual actions in the shared virtual environment.